SPECIFICATION 



TO ALL WHOM IT MAY CONCERN: 

BE IT KNOWN THAT WE, Shigeru Nishio, a 
citizen of Japan residing at Kawasaki, Japan, Hiroshi 
Daiku, a citizen of Japan residing at Kawasaki, Japan 
and Asao Kokubo, a citizen of Japan residing at Kawasaki, 
Japan have invented certain new and useful improvements in 



CIRCUIT AND METHOD FOR CONTOUR ENHANCEMENT 



of which the following is a specification:- 
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TITLE OF THE INVENTION 

CIRCUIT AND METHOD FOR CONTOUR ENHANCEMENT 

CROSS-REFERENCE TO RELATED APPLICATIONS 

5 The present application is based upon and 

claims the benefit of priority from the prior 
Japanese Patent Application No. 2002-216848 filed on 
July 25, 2002, and Japanese priority application No. 
2002-316078 filed on October 30, 2002, with the 
10 Japanese Patent Office, the entire contents of which 
are incorporated herein by reference. 

BACKGROUND OF THE INVENTION 

1, Field of the Invention 

15 The present invention generally relates to 

semiconductor integrated circuits and methods for 
image processing, and particularly relates to a 
semiconductor integrated circuit and a method for 
enhancing contours in image signals supplied from an 

20 image sensor. 

2 . Description of the Related Art 

A CMOS image sensor includes a reception 
section in which a plurality of photo-diodes are 
arranged in a matrix array. These photo-diodes each 

25 correspond to a pixel of a captured image. Incident 
light is subjected to photoelectric conversion on a 
pixel-by-pixel basis, and the electric charge 
obtained by the photoelectric conversion is 
accumulated in an electric charge accumulation 

30 section for reading of data. Data of each pixel is 
supplied to a processing circuit such as an image 
processor for such processing as RGB conversion, 
contour enhancement, and format conversion, followed 
by being supplied to a memory or to an image display 

35 apparatus or the like. 

In order to enhance image contours, 
conventionally, uniform image enhancement is applied 
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unvaryingly to all the pixels of an image. Namely, 
differences of image data levels with adjacent 
pixels are obtained with respect to every pixel of 
the image, and these differences are enlarged by a 
5 predetermined factor so as to enlarge differences of 
image data levels between adjacent pixels. 

A Japanese Patent Application Publication 
No. 5-22601 discloses a process that avoids an 
artifact of multiple contours without degrading the 

10 performance of contour enhancement. This process 
obtains a first-order derivation by calculating 
differentials in a predetermined direction while 
providing a dead zone in which no processing is 
performed, and further obtains a second-order 

15 derivation in this predetermined direction, followed 
by performing an averaging process to generate an 
edge-enhanced signal . 

When all the pixels of image data are 
subjected to image enhancement, however, image noise 

20 is enhanced altogether, resulting in undesirable 
degradation of image quality. 

Accordingly, there is a need for a method 
and a circuit for contour enhancement that can 
easily generate contour-enhanced images without 

25 degrading image quality. 

SUMMARY OF THE INVENTION 

It is a general object of the present 
invention to provide a method and a circuit for 

30 contour enhancement that substantially obviates one 
or more problems caused by the limitations and 
disadvantages of the related art. 

Features and advantages of the present 
invention will be presented in the description which 

35 follows, and in part will become apparent from the 

description and the accompanying drawings, or may be 
learned by practice of the invention according to 
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the teachings provided in the description. Objects 
as well as other features and advantages of the 
present invention will be realized and attained by a 
method and a circuit for contour enhancement 
5 particularly pointed out in the specification in 
such full, clear, concise, and exact terms as to 
enable a person having ordinary skill in the art to 
practice the invention. 

To achieve these and other advantages in 

10 accordance with the purpose of the invention, the 
invention provides a semiconductor integrated 
circuit, including a differential calculating unit 
which obtains a differential between a value of a 
pixel of interest and values of surrounding pixels 

15 contained in an image signal supplied from an image 
sensor, a dead-zone generating unit which defines a 
predetermined range of pixel values, and a 
comparison unit which checks whether the 
differential falls outside the predetermined range, 

20 wherein contour enhancement is applied to the pixel 
of interest in response to a determination by the 
comparison unit that the differential falls outside 
the predetermined range. 

In the semiconductor integrated circuit as 

25 described above, contour enhancement is not 

unvaryingly applied to the entirety of the image 
data, but is applied to only those image portions 
which are identified as requiring contour 
enhancement. This achieves efficient contour 

30 enhancement that avoids unnecessary enhancement of 
noise . 

The calculation of the differential and 
the contour enhancement may be performed only with 
respect to the green-color component among a 
35 plurality of color components or only with respect 
to the luminance component. With this provision, a 
line-delay memory needs to be provided only for the 



-4- 



green-color component or the luminance component, 
thereby achieving a relatively small circuit size. 

Further, a method of enhancing contours 
includes the steps of obtaining a differential 
5 between a value of a pixel of interest and values of 
surrounding pixels contained in an image signal 
supplied from an image sensor, defining a 
predetermined range of pixel values, checking 
whether the differential falls outside the 

10 predetermined range, and applying contour 

enhancement to the pixel of interest in response to 
a determination that the differential falls outside 
the predetermined range. 

Other objects and further features of the 

15 present invention will be apparent from the 
following detailed description when read in 
conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

20 Fig.l is a block diagram of an image 

processor for performing image processing including 
contour enhancement according to the present 
invention ; 

Fig. 2 is a block diagram of a contour 
25 enhancing unit according to a first embodiment of 
the present invention; 

Fig. 3 is an illustrative drawing for 
explaining the pixel of interest subjected to 
contour enhancement; and 
30 Fig. 4 is a block diagram of the contour 

enhancing unit according to a second embodiment of 
the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

35 In the following, embodiments of the 

present invention will be described with reference 
to the accompanying drawings . 



-5- 



Fig.l is a block diagram of an image 
processor for performing image processing including 
contour enhancement according to the present 
invention . 

5 The image processor of Fig.l is 

implemented as an LSI chip, and includes a defect 
correcting unit 11, a RGB conversion unit 12 , a 
white balancing unit 13, a contour enhancing unit 14, 
a gamma correction unit 15, and a format conversion 
10 unit 16. 

The defect correcting unit 11 receives 
image signals from an image sensor such as a CCD 
(charge coupled device) , and carries out defect 
correction. An image sensor inevitably includes 

15 pixels that are defective and thus cannot properly 
operate. Such pixels appear as fixed black dots or 
white dots. The defect correcting unit 11 is 
provided for the purpose of correcting image data of 
these defective pixels. The image signals having 

20 undergone the defective pixel correction are 

supplied from the defect correcting unit 11 to the 
RGB conversion unit 12. The RGB conversion unit 12 
outputs image signals by obtaining color data of 
each pixel based on color information of the RGB 

25 Bayer array. The image signals output from the RGB 
conversion unit 12 are then subjected to white 
balancing by the white balancing unit 13. 

After this, the contour enhancing unit 14 
for performing contour enhancement of the present 

30 invention carries out a process for enhancing 

contours of the supplied image data. The gamma 
correction unit 15 applies a process for correcting 
gamma characteristics of the output apparatus . 
Finally, the format conversion unit 16 converts the 

35 image signals into a format acceptable to a 

subsequent processing stage, and the converted image 
signals are output. 
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Fig.2 is a block diagram of the contour 
enhancing unit 14 according to a first embodiment of 
the present invention. 

The contour enhancing unit 14 of Fig. 2 
5 includes a differential calculating unit 21 , a dead- 
zone generating unit 22, a comparison unit 23 , and 
an enhancement value generating unit 24. The 
differential calculating unit 21 obtains 
differentials of pixel values between a pixel of 

10 interest and surrounding pixels. The dead-zone 

generating unit 22 obtains a predetermined range of 
pixels that is defined by an upper limit and a lower 
limit. The comparison unit 23 checks whether the 
differentials supplied from the differential 

15 calculating unit 21 fall into the predetermined 

range indicated by the dead-zone generating unit 22. 
The enhancement value generating unit 24 generates a 
correction value (enhancement value) for correcting 
the value of the pixel of interest for the purpose 

20 of contour enhancement in response to a 

determination that the differentials between the 
pixel of interest and the surrounding pixels do not 
fall within the predetermined range. The generated 
correction value is added to the pixel value of the 

25 input RGB signals. 

Fig. 3 is an illustrative drawing for 
explaining the pixel of interest subjected to 
contour enhancement . 

In the typical output of an image sensor, 

30 each pixel does not have the data indicative of 
three R, G, and B colors. Rather, each pixel 
represents one pre-assigned color among the RGB 
colors such as in the RGB Bayer array. Based on the 
RGB Bayer array, the RGB conversion unit 12 performs 

35 a process for assigning the three RGB colors to each 
pixel. Accordingly, the image data subjected to 
contour enhancement by the contour enhancing unit 14 
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has all the three RGB colors already assigned to 
each pixel. 

In the present invention, a pixel of 
interest p[x, y] is given, and the four neighboring 
5 pixels p[x, y-1], p[x-l, y] , p[x+l, y] , and p[x, 

y+1] are taken into consideration to perform contour 
enhancement in respect of the pixel of interest. 

Further, the present invention does not 
perform uniform contour enhancement unvaryingly over 

10 the entirety of image data, but carries out contour 
enhancement only on image portions that are 
identified as needing contour enhancement according 
to the determination by the comparison unit 23. 
Here, the RGB colors and a luminance component Y are 

15 related as: 

Y = 0.299 x R + 0.587 x G + 0.114 x B. (1) 
As is shown in this equation, the G component, among 
the three colors, contains almost 60% of the total 
luminance component. When a decision has to be made 

20 as to whether contour enhancement needs to be 

applied, therefore, only the G component may be 
referred to, thereby achieving an efficient check. 

In the first embodiment of contour 
enhancement, the G component is utilized to 

25 determine whether contour enhancement needs to be 
applied to a pixel of interest. 

In respect of G[x, y] of a pixel of 
interest, a difference between this G component and 
an average of G components of the surrounding pixels 

30 is computed. A check is then made as to whether the 
following condition is satisfied, 
lowjimit <(G[x-1 J y]+G[x+1,y]+G[x,y-1]+G[x,y+1])/4-G[x,y]< maxjimit (2) 
Here, the lower limit low_limit and the upper limit 
max_limit are generated by the dead-zone generating 

35 unit 22. The differential calculating unit 21 

generates the difference between the G component of 
the pixel of interest and the average of G 
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components of the surrounding pixels for comparison 
with the upper and lower limits. The comparison 
unit 23 determines whether the condition (2) is 
satisfied. The lower limit low_limit and the upper 
5 limit upper_limit may be obtained by use of the 
values of the surrounding pixels as follows 

low_limit = L_limit x pixel_value + L_offset 
max_limit = M_limit x pixel_value - M_offset 
Here, pixel_value may be an average or median of the 

10 values of predetermined surrounding pixels. L_limit 
and L_offset are the values for defining the lower 
limit of the dead zone. M_limit and M_offset are 
the values for defining the upper limit of the dead 
zone. L_limit and M_limit may be the same value, or 

15 may be set to different values. L_offset and 

M_offset may be the same value, or may be set to 
different values. 

When the condition (2) is satisfied, it is 
decided that contour enhancement is not performed. 

20 The enhancement value generating unit 24 does not 
generate a correction value (enhancement value) in 
this case. As a result, the pixel of interest of 
the input signals will have the same R, G, and B 
color values as before, i.e., 

25 R[x, y] = R[x, y] 

G[x, y] = G[x, y] 
B[x, y] = B[x, y] . 
When the condition (2) is not satisfied, i.e., when 
the difference between the G component of the pixel 

30 of interest and the average of G components of the 

surrounding pixels falls within the range defined by 
the upper and lower limits, the enhancement value 
generating unit 24 generates a correction value 
(enhancement value) . The correction value is then 

35 added to (or rather "subtracted from" as a form of 
expression) each of the R, G, and B colors of the 
pixel of interest of the input signals. 
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R[x,y] = R[x,y] - Rgain x ((G[x-1 l yl+G[x+1 l y]+G[x,y-1]+G[x l y+1J)/4 - G[x,y]) 
G[x,y] = G[x,y] - Ggain x ((Gtx-l^+Gfx+^yl+Gfx.y-ll+G^y+IDM - G[x,y]) 
B[x,y] = B[x,y] - Bgain x ((Glx-ljl+Glx+l.yl+G^y-ll+Gtx.y+IDM - G[x,y]) 

Here, the second term on the right-hand side of each 
5 equation is the correction value (enhancement value) . 
Rgain, Ggain, and Bgain are gain factors for 
defining the degree of contour enhancement with 
respect to R, G, and B, respectively. It suffices 
to have the gain factors that are more than zero, 

10 and the gain factors may be set to 3.0, for example. 

In the process as described above, a 
correction value that is obtained from the G 
components of the pixel of interest and the 
surrounding pixels is applied to another color (e.g., 

15 the R color) of the pixel of interest for the 

purpose of enhancing contours of this color (i.g., 
the R color) . It is conceivable that a correction 
value obtained from R components of the pixel of 
interest and the surrounding pixels is used in 

20 contour enhancement of the R color. In order to 
achieve this, however, a memory such as a FIFO 
memory for 2-line delay needs to be provided for 
every single color, resulting in the circuit size of 
the differential calculating unit 21 being expanded. 

25 In the embodiment described above, correction values 
for contour enhancement are limited to only the G 
component, so that the line memory needs to be 
provided only for the G component. This 
configuration requires a relatively small circuit 

30 size. 

In the embodiment as described above, 
contour enhancement is not unvaryingly applied to 
the entirety of the image data, but is applied to 
only those image portions which are identified as 
35 requiring contour enhancement. This achieves 
efficient contour enhancement that avoids 
unnecessary enhancement of noise. 
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In the following, a variation of the first 
embodiment of the present invention will be 
described . 

The differential calculating unit 21 
5 calculates differentials between the G component of 
the pixel of interest and the G components of the 
surrounding pixels as follows. 

diff_l = G[x-1, y] - G[x, y] 
diff_2 = G[x+1, y] - G[x, y] 
10 diff_3 = G[x, y-1] - G[x, y] 

diff_4 = G[x, y+1] - G[x, y] 
Then, low_limit and max_limit are generated by the 
dead-zone generating unit 22 for comparison by the 
comparison unit 23 as follows. 
15 low_limit < diff_l < max_limit (3) 

low_limit < diff_2 < max__limit (4) 
low_limit < diff_3 < max_limit (5) 
low_limit < diff_4 < max_limit (6) 
Contour enhancement is not performed if all the 
20 conditions (3) through (6) are satisfied. In this 
case, the enhancement value generating unit 24 does 
not generate a correction value (enhancement value) . 
The values of the pixel of interest of input signals 
will thus stay the same with respect to each of R, G, 
2 5 and B , i.e., 

R[x, y] = R[x, y] 
G[x, y] = G[x, y] 
B[x, y] = B[x, y] . 
Further, contour enhancement is not performed either 
30 if the largest one of diff_l through diff_4 has an 

absolute value identical to that of the smallest one 
of diff_l through diff_4. This condition 
corresponds to a border case that comes between a 
case in which the value of the pixel of interest 
35 should be increased and a case in which the value of 
the pixel of interest should be decreased. Because 
of this, no correction is made to the pixel of 
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interest . 

If one of the conditions (3) through (6) 
is not satisfied, diff_max is identified as having 
the largest absolute value among diff_l through 
5 diff_4, and the enhancement value generating unit 2 4 
generates a correction value (enhancement value) as 
follows. The generated correction value is then 
added to (or rather "subtracted from" as a form of 
expression) each of the R, G, and B colors of the 

10 pixel of interest of the input signals. 
R[x,y] = R[x,y] - Rgain x diffjnax 
G[ x >y] = G[ x >y] - Ggain x diffjnax 
B[x,y] = B[x,y] - Bgain x diffjnax 
Here, the second term on the right-hand side of each 

15 equation is the correction value (enhancement value) . 
Rgain, Ggain, and Bgain are gain factors for 
defining the degree of contour enhancement with 
respect to R, G, and B, respectively. It suffices 
to have the gain factors that are more than zero., 

20 and the gain factors may be set to 0.8, for example. 

In the variation of the first embodiment 
described above, a line-delay memory should be 
provided only for the G component in the same manner 
as in the first embodiment, thereby achieving a 

25 relatively small circuit size. Further, contour 
enhancement is not unvaryingly applied to the 
entirety of the image data, but is applied to only 
those image portions which are identified as 
requiring contour enhancement. This achieves 

30 efficient contour enhancement that avoids 

unnecessary enhancement of noise. In comparison 
with the first embodiment, an effect of contour 
enhancement will be larger since the differential 
having the largest absolute value among diff_l 

35 through diff_4 is selected for use in generating the 
correction value (enhancement value) . At a slope 
where the image level changes monotonously, the 
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first embodiment tends to generate a small 
correction value (enhancement value) . On the other 
hand, the variation of the first embodiment can 
guarantee a relatively large correction value 
5 (enhancement value) , thereby producing a sufficient 
contour enhancement effect. 

Fig. 4 is a block diagram of the contour 
enhancing unit 14 according to a second embodiment 
of the present invention. 

10 The contour enhancing unit 14 of Fig. 4 

includes a Y-signal generating unit 30, a 
differential calculating unit 31, a dead-zone 
generating unit 32, a comparison unit 33, and an 
enhancement value generating unit 34. The Y-signal 

15 generating unit 30 generates a luminance signal Y 

from the input RGB signals based on the equation (1) 
previously presented. In the second embodiment, 
contour enhancement is performed by use of the 
luminance component Y in place of the G component. 

20 The differential calculating unit 31 

obtains differentials of pixel values between a 
pixel of interest and surrounding pixels. The dead- 
zone generating unit 32 obtains a predetermined 
range of pixels that is defined by an upper limit 

25 and a lower limit. The comparison unit 33 checks 
whether the differentials supplied from the 
differential calculating unit 31 fall into the 
predetermined range indicated by the dead-zone 
generating unit 32. The enhancement value 

30 generating unit 34 generates a correction value 

(enhancement value) for correcting the value of the 
pixel of interest for the purpose of contour 
enhancement in response to a determination that the 
differentials between the pixel of interest and the 

35 surrounding pixels do not fall within the 

predetermined range. The generated correction value 
is added to the pixel value of the input RGB signals. 
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In respect of a Y component Y[x, y] of a 
pixel of interest, a difference between this Y 
component and an average of Y components of the 
surrounding pixels is computed. A check is then 
5 made as to whether the following condition is 
satisfied . 

lowjimit <(Y[x-1,y]+Y[x+1,y]+Y[x,y-1]+Y[x,y+1])/4-Y[x ) y]< maxjimit (7) 

Here, the lower limit low_limit and the upper limit 
max_limit are generated by the dead-zone generating 

10 unit 32 based on the Y components of neighboring 
pixels. The differential calculating unit 31 
generates the difference between the Y component of 
the pixel of interest and the average of Y 
components of the surrounding pixels for comparison 

15 with the upper and lower limits. The comparison 
unit 33 determines whether the condition (7) is 
satisfied . 

When the condition (7) is satisfied, it is 
decided that contour enhancement is not performed. 

20 The enhancement value generating unit 34 does not 
generate a correction value (enhancement value) in 
this case. As a result, the pixel of interest of 
the input signals will have the same R, G, and B 
color values as before, i.e., 

25 R[x, y] = R[x, y] 

G[x, y] = G[x, y] 
B[x, y] = B[x, y] . 
When the condition (7) is not satisfied, i.e., when 
the difference between the Y component of the pixel 

30 of interest and the average of Y components of the 

surrounding pixels falls within the range defined by 
the upper and lower limits, the enhancement value 
generating unit 34 generates a correction value 
(enhancement value) . The correction value is then 

35 added to (or rather "subtracted from" as a form of 
expression) each of the R, G, and B colors of the 
pixel of interest of the input signals. 
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R[x,y] = R[x,y] - Rgain x ((Y[x-1,y]+Y[x+1,y]+Y[x,y-1]+Y[x,y+1])/4 - Y[x,y]) 
G[x,y] = G[x,y] - Ggain x ((Y[x-1 jl+Yfx+l.yl+Yfx.y-ll+Ylx.y+llJM - Y[x,y]) 
B[x,y] = B[x,y] - Bgain x ((Ytx-l.yl+Yfx+l.yl+Ylx.y-ll+Ytx.y+IDM - Y[x,y]) 

Here, the second term on the right-hand side of each 
5 equation is the correction value (enhancement value) . 
Rgain, Ggain, and Bgain are gain factors for 
defining the degree of contour enhancement with 
respect to R, G, and B, respectively. It suffices 
to have the gain factors that are more than zero. 

10 In the process as described above, a 

correction value that is obtained from the Y 
components of the pixel of interest and the 
surrounding pixels is applied to a given color (e.g., 
the R color) of the pixel of interest for the 

15 purpose of enhancing contours of this color (i.g., 
the R color) . With only the Y component being used 
in generating the correction value for contour 
enhancement, the line delay memory needs to be 
provided only for the Y component in the 

20 differential calculating unit 31. This 

configuration requires a relatively small circuit 
size . 

In the embodiment as described above, 
contour enhancement is not unvaryingly applied to 

25 the entirety of the image data, but is applied to 
only those image portions which are identified as 
requiring contour enhancement. This achieves 
efficient contour enhancement that avoids 
unnecessary enhancement of noise. Further, this 

30 embodiment achieves contour enhancement using 

accurate luminance information by use of a circuit 
having a relatively small size. 

In the following, a variation of the 
second embodiment of the present invention will be 

35 described. 

The differential calculating unit 31 
calculates differentials between the Y component of 
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the pixel of interest and the Y components of the 
surrounding pixels as follows. 

diff_l = Y[x-1, y] - Y[x, y] 

diff_2 = Y[x+1, y] - Y[x, y] 
5 diff_3 = Y[x, y-1] - Y[x, y] 

diff_4 = Y[x, y+1] - Y[x, y] 
Then, low_limit and max_limit are generated by the 
dead-zone generating unit 32 for comparison by the 
comparison unit 33 as follows. 
10 low_lirnit < diffy_l < max_limit (8) 

low_limit < diffy_2 < max_limit (9) 

low_limit < diffy_3 < max_limit (10) 

low_limit < diffy__4 < max_limit (11) 
Contour enhancement is not performed if all the 
15 conditions (8) through (11) are satisfied. In this 
case, the enhancement value generating unit 34 does 
not generate a correction value (enhancement value) . 
The values of the pixel of interest of input signals 
will thus stay the same with respect to each of R, G , 
2 0 and B , i.e., 

R[x, y] = R[x, y] 

G[x, y] = G[x, y] 

B[x, y] = B[x f y] . 
Further, contour enhancement is not performed either 
25 if the largest one of diffy__l through diffy_4 has an 
absolute value identical to that of the smallest one 
of diffy_l through diffy__4. This condition 
corresponds to a border case that comes between a 
case in which the value of the pixel of interest 
30 should be increased and a case in which the value of 
the pixel of interest should be decreased. Because 
of this, no correction is made to the pixel of 
interest . 

If one of the conditions (8) through (11) 
35 is not satisfied, diffy_max is identified as having 
the largest absolute value among diffy_l through 
diffy_4, and the enhancement value generating unit 
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34 generates a correction value (enhancement value) 
as follows. The generated correction value is then 
added to (or rather "subtracted from" as a form of 
expression) each of the R, G, and B colors of the 
5 pixel of interest of the input signals. 
R[x,y] = R[x,y] - Rgain x diffyjnax 
G[x,y] = G[x,y] - Ggain x diffyjnax 
B[x,y] = B[x,y] - Bgain x diffyjnax 
Here, the second term on the right-hand side of each 

10 equation is the correction value (enhancement value) . 
Rgain, Ggain, and Bgain are gain factors for 
defining the degree of contour enhancement with 
respect to R, G, and B, respectively. It suffices 
to have the gain factors that are more than zero, 

15 and the gain factors may be set to 0.8, for example. 

In the variation of the second embodiment 
described above, a line-delay memory should be 
provided only for the Y component in the same manner 
as in the second embodiment, thereby achieving a 

20 relatively small circuit size. Further, contour 

enhancement using accurate luminance information is 
attained. In comparison with the second embodiment, 
the variation of the second embodiment can guarantee 
a relatively large correction value (enhancement 

25 value) , thereby producing a sufficient contour 
enhancement effect . 

Further, the present invention is not 
limited to these embodiments, but various variations 
and modifications may be made without departing from 

30 the scope of the present invention. 



